Mixed Media Search in Image Databases Using Text and Visual Similarity

This paper presents a methodology for interacting with images using both text and visual features. When images are entered into the database, they are associated with a piece of text (typically a web page), which is used to derive a lexical index for the image. At the same time, visual features are computed. The similarity between two images is then a combination between their visual similarity and the similarity of the texts attached to them. The user can intervene on either one of these similarities with repercussion on both the lexical similarity between terms and the semantic similarity between images.